Search for: All records

Although Federated Learning (FL) enables global model training across clients without compromising their raw data, due to the unevenly distributed data among clients, existing Federated Averaging (FedAvg)-based methods suffer from the problem of low inference performance. Specifically, different data distributions among clients lead to various optimization directions of local models. Aggregating local models usually results in a low-generalized global model, which performs worse on most of the clients. To address the above issue, inspired by the observation from a geometric perspective that a well-generalized solution is located in a flat area rather than a sharp area, we propose a novel and heuristic FL paradigm named FedMR (Federated Model Recombination). The goal of FedMR is to guide the recombined models to be trained towards a flat area. Unlike conventional FedAvg-based methods, in FedMR, the cloud server recombines collected local models by shuffling each layer of them to generate multiple recombined models for local training on clients rather than an aggregated global model. Since the area of the flat area is larger than the sharp area, when local models are located in different areas, recombined models have a higher probability of locating in a flat area. When all recombined models are located in the same flat area, they are optimized towards the same direction. We theoretically analyze the convergence of model recombination. Experimental results show that, compared with state-of-the-art FL methods, FedMR can significantly improve the inference accuracy without exposing the privacy of each client. 

Human-generated Spatial-Temporal Data (HSTD), represented as trajectory sequences, has undergone a data revolution, thanks to advances in mobile sensing, data mining, and AI. Previous studies have revealed the effectiveness of employing attention mechanisms to analyze massive HSTD. However, traditional attention models face challenges when managing lengthy and noisy trajectories as their computation comes with large memory overheads. Furthermore, attention scores within HSTD trajectories are sparse (i.e., most of the scores are zeros), and clustered with varying lengths (i.e., consecutive tokens clustered with similar scores). To address these challenges, we introduce an innovative strategy named Memory-efficient Trajectory Attention (MeTA). We leverage complicated spatial-temporal features (e.g., traffic speed, proximity to PoIs) and design an innovative feature-based trajectory partition technique to shrink trajectory length. Additionally, we present a learnable dynamic sorting mechanism, with which attention is only computed between sub-trajectories that have prominent correlations. Empirical validations using real-world HSTD demonstrate that our approach not only yields competitive results but also significantly lowers memory usage compared with state-of-the-art methods. Our approach presents innovative solutions for memory-efficient trajectory attention, offering valuable insights for handling HSTD efficiently. 

To explore the connection between brain and behavior in engineering design, this study measured the change in neurocognition of engineering students while they developed concept maps. Concept maps help designers organize complex ideas by illustrating components and relationships. Student concept maps were graded using a pre-established scoring method and compared to their neurocognitive activation. Results show significant correlations between performance and neurocognition. Concept map scores were positively correlated with activation in students’ prefrontal cortex. A prominent sub-region was the right dorsolateral prefrontal cortex (DLPFC), which is generally associated with divergent thinking and cognitive flexibility. Student scores were negatively correlated with measures of brain network density. The findings suggest a possible neurocognitive mechanism for better performance. More research is needed to connect brain activation to the cognitive activi-ies that occur when designing but these results provide new evidence for the brain functions that support the development of complex ideas during design. 

The matrix valued exponential function can be used for time-stepping numerically stiff discretization, such as the discontinuous Galerkin method but this approach is expensive as the matrix is dense and necessitates global communication. In this paper, we propose a local low-rank approximation to this matrix. The local low-rank construction is motivated by the nature of wave propagation and costs significantly less to apply than full exponentiation. The accuracy of this time stepping method is inherited from the exponential integrator and the local property of it allows parallel implementation. The method is expected to be useful in design and inverse problems where many solves of the PDE are required. We demonstrate the error convergence of the method for the one-dimensional (1D) Maxwell’s equation on a uniform grid. 

In this paper, we explored changes in brain states over time while designers were generating concepts. Participants either used morphological analysis or TRIZ to develop a design concept for two design tasks. While designing, participants’ brain activation in their prefrontal cortex (PFC) was monitored with a functional Near Infrared Spectroscopy machine. To identify variation in brain states, we analyzed changes in brain networks. Using k-mean clustering to classify brain networks for each task revealed four brain network patterns. While using morphological analysis, the occurrence of each pattern was similar along the design steps. For TRIZ, some brain states dominated depending on the design step. Drain states changes suggests that designers alternate engaging certain subregions of the PFC. This approach to studying brain behavior provides a more granular understanding of the evolution of design brain states over time. Findings add to the growing body of research exploring design neurocognition. 

In this paper, we explored changes in brain states over time while designers were generating concepts. Participants either used morphological analysis or TRIZ to develop a design concept for two design tasks. While designing, participants’ brain activation in their prefrontal cortex (PFC) was monitored with a functional Near Infrared Spectroscopy machine. To identify variation in brain states, we analyzed changes in brain networks. Using k-mean clustering to classify brain networks for each task revealed four brain network patterns. While using morphological analysis, the occurrence of each pattern was similar along the design steps. For TRIZ, some brain states dominated depending on the design step. Drain states changes suggests that designers alternate engaging certain subregions of the PFC. This approach to studying brain behavior provides a more granular understanding of the evolution of design brain states over time. Findings add to the growing body of research exploring design neurocognition. 

The research presented in this paper explores the effect of concept maps on students’ neurocognition when constructing engineering problem statements. In total, 66 engineering students participated in the experiment. Half of the students were asked to create a concept map illustrating all of the systems and stakeholders represented in a building on campus. The other half of students were not asked to draw a concept map. Both groups were then asked to construct an engineering problem statement about improvements to the building. While performing the problem statement task, their neurocognitive activation in the prefrontal cortex (PFC) was measured using a non-intrusive neuroimaging technique called functional near-infrared spectroscopy. The students that were asked to complete the concept mapping task required less cognitive effort to formulate and analyze their problem statements. The specific regions that were less activated were regions of the brain generally associated with working memory and problem evaluation. These results provide new insight into the changes in mental processing that occurs when using tools like concept maps and may provide helpful techniques for students to structure engineering problems. 

The research presented in this paper explores features of temporal design neurocognition by comparing regions of activation in the brain during concept generation. A total of 27 engineering graduate students used brainstorming, morphological analysis, and TRIZ to generate concepts to design problems. Students' brain activation in their prefrontal cortex (PFC) was measured using functional near-infrared spectroscopy (fNIRS). Temporal activations were compared between techniques. When using brainstorming and morphological analysis, highly activated regions are consistently situated in the medial and right part of the PFC over time. For both techniques, the temporal neuro-physiological patterns are similar. Cognitive functions associated to the medial and right part of the PFC suggest an association with divergent thinking and adaptive decision making. In contrast, highly activated regions over time when using TRIZ appear in the medial or the left part of the prefrontal cortex, usually associated with goal directed planning. 

The Theory of Inventive Problem Solving (TRIZ) method and toolkit provides a well-structured approach to support engineering design with pre-defined steps: interpret and define the problem, search for standard engineering parameters, search for inventive principles to adapt, and generate final solutions. The research presented in this paper explores the neurocognitive differences of each of these steps. We measured the neuro-cognitive activation in the prefrontal cortex (PFC) of 30 engineering students. Neuro-cognitive activation was recorded while students completed an engineering design task. The results show a varying activation pattern. When interpreting and defining the problem, higher activation is found in the left PFC, generally associated with goal directed planning and making analytical judgement when interpreting and defining the problem. Neuro-cognitive activation shifts to the right PFC during the search process, a region usually involved in exploring the problem space. During solution generation more activation occurs in the medial PFC, a region generally related to making associations. The findings offer new insights and evidence explaining the dynamic neuro-cognitive activations when using TRIZ in engineering design.